Anticipatory On-Line Planning

We consider the problem of on-line continual planning, in which additional goals may arrive while plans for previous goals are still executing and plan quality depends on how quickly goals are achieved. This is a challenging problem even in domains with deterministic actions. One common and straightforward approach is reactive planning, in which plans are synthesized when a new goal arrives. In this paper, we adapt the technique of hindsight optimization from online scheduling and probabilistic planning to create an anticipatory on-line planning algorithm. Using an estimate of the goal arrival distribution, we sample possible futures and use a deterministic planner to estimate the value of taking possible actions at each time step. Results in two benchmark domains based on unmanned aerial vehicle planning and manufacturing suggest that an anticipatory approach yields a superior planner that is sensitive not only to which action should be